Comparing changes

…64 (#22458) Add `write_u16_le` and `write_u16_be`, and also `read_` functions and variants for `u32` and `u64`. These are very similar to the existing signed variants. Also add `write_i8` and `read_i8`. Reading a signed byte previously required `i8(u8(b[n]))`, since `i8(b[n])` raises OverflowError for values above 127, so having dedicated functions is useful (unlike for `u8`). Work on mypyc/mypyc#1226.

`run_analysis` keeps a Python `set` for every basic block, and a new one for every op whose gen/kill sets aren't empty. In large functions these sets have thousands of items, so the data-flow analyses take O(ops * values) time and memory. The must-defined and borrowed-arguments analyses are the worst, since a must analysis starts every block from a copy of the full universe. This makes the refcount transform very slow for large functions, such as generated code or long module top levels. This PR represents the sets as `int` bit masks: - `BitIndex` assigns a bit position to each value an analysis sees. - The worklist algorithm works on masks, so union, intersection and difference are integer operations. Since ints are immutable, an op whose gen/kill sets are empty shares the previous result, as before. - The gen/kill masks of each op are computed once, instead of running the visitor again in the op-level pass. - The results are `BitSet` objects that implement `collections.abc.Set`, so the code using the results only needed annotation changes (`set[Value]` -> `AbstractSet[Value]` in refcount.py). - `analyze_live_regs`, `analyze_borrowed_arguments` and `analyze_must_defined_regs` take an optional `index`. The refcount transform shares one index between the three, because it combines their results (e.g. `live - live - borrowed` after a branch); with a shared index these operations are integer operations too. With different indexes, `BitSet` still works and, like `set`, iterates over the smaller operand. - Reflected operations (`set - bitset`, `set & bitset`, `set | bitset`) return a real `set`, since typeshed says the result is a `set` and compiled code checks this. `find_always_defined_attributes` relies on it. ## Benchmark A function with n statements of this form: ```python a{i} = xs[{i % 7}] + {i} if a{i}: t = a{i} ``` Each analysis timed separately, then `insert_ref_count_opcodes` (interpreted). Max RSS is for the whole process: | n | ops / blocks | | live | borrowed | must-defined | refcount transform | max RSS | |---|---|---|---|---|---|---|---| | 1000 | 9k / 2k | master | 0.02s | 0.13s | 0.61s | 0.83s | 1.2 GiB | | | | this PR | 0.02s | 0.00s | 0.01s | 0.09s | 73 MiB | | 2000 | 18k / 4k | master | 0.03s | 0.67s | 2.2s | 3.5s | 4.7 GiB | | | | this PR | 0.06s | 0.01s | 0.02s | 0.23s | 109 MiB | | 4000 | 36k / 8k | master | 0.09s | 2.2s | 93s | 28–84s (swapping) | 16 GiB | | | | this PR | 0.19s | 0.02s | 0.05s | 0.63s | 225 MiB | Liveness is a bit slower, because live sets are small and a mask grows with the number of values. It is still a small part of the total. On ordinary code, compiling `mypy/checker.py`, `mypy/checkexpr.py`, `mypy/semanal.py` and `mypy/messages.py` with the analysis and transform modules compiled by mypyc: | | master | this PR | |---|---|---| | `insert_ref_count_opcodes` | 1.59–1.73s | 0.94–0.95s | | `insert_uninit_checks` | 0.28–0.29s | 0.21s | ## Testing - The generated C for those four modules is identical to master, both with interpreted and compiled analysis modules, and the IR for the benchmark functions is identical. - The full mypyc test suite passes interpreted. The IR, refcount, exceptions, always-defined, spill and run tests also pass with `dataflow.py` and its users compiled by mypyc. - New `mypyc/test/test_bitset.py` covers `BitSet` operations: same index, different index, mixed with `set`, and the types the reflected operations return. This is independent of #22425 (which removes temporaries from the must-defined universe). The two merge without conflicts and help in different ways: #22425 makes the sets smaller, this PR makes each item cost a bit instead of a hash table entry.

#22453) Fixes mypyc/mypyc#1253 The `tp_hash` slot of a native class converted the value returned by `__hash__` with `CPyTagged_AsSsize_t()`, so an int that doesn't fit in `Py_hash_t` made `hash()` raise `OverflowError`. CPython's `slot_tp_hash`, which a class defined in Python uses, accepts any int and hashes one that doesn't fit again with `int.__hash__`. `generate_hash_wrapper()` now does the same: - A short int is still handled inline, with -1 changed to -2 as before. - A long int goes through the new `CPy_HashFromResult()` in `generic_ops.c`, which does what `slot_tp_hash` does with the result: an int that fits in `Py_hash_t` is used as is, a larger one is hashed again, and anything that isn't an int raises `TypeError: __hash__ method should return an integer`. - Other return types are boxed if needed and passed to `CPy_HashFromResult()`. With `-> Any`, a result that isn't an int used to raise `TypeError: an integer is required`. A `__hash__` declared to return a native int type such as `i64` segfaulted (or raised `SystemError` if it returned 0), because the slot passed the C value to `PyLong_AsSsize_t()` as an object pointer, but now this works. The new run test `testDundersHashResult` covers `-> int` and `-> Any` with values that fit in `Py_hash_t`, -1, values that don't fit, a long int that fits only on 64-bit platforms (2**62), `True` and results that aren't ints, plus an `-> i64` method that returns -1. The expected results are the same on 32-bit platforms.

Navigation Menu

Repository navigation

Comparing changes

Open a pull request

Commits on Nov 13, 2026

This comparison is taking too long to generate.

Uh oh!